Recently, Witten showed that there is a natural action of the group SL(2,Z) on the space of 3 dimensional conformal field theories with U(1) global symmetry and a chosen coupling of the symmetry current to a background gauge field on a 3-fold N. He further argued that, for a class of conformal field theories, in the nearly Gaussian limit, this SL(2,Z) action may be viewed as a holographic image of the well-known SL(2,Z) Abelian duality of a pure U(1) gauge theory on AdS-like 4-folds M bounded by N, as dictated by the AdS/CFT correspondence. However, he showed that explicitly only for the generator T; for the generator S, instead, his analysis remained conjectural. In this paper, we propose a solution of this problem. We derive a general holographic formula for the nearly Gaussian generating functional of the correlators of the symmetry current and, using this, we show that Witten's conjecture is indeed correct when N=S^3. We further identify a class of homology 3-spheres N for which Witten's conjecture takes a particular simple form.